<html>
<head><meta charset="utf-8"><title>Can async {} futures be Sync/Send more often? · wg-async-foundations · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/index.html">wg-async-foundations</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html">Can async {} futures be Sync/Send more often?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="221358731"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221358731" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221358731">(Jan 01 2021 at 16:43)</a>:</h4>
<p>Futures can sometimes end up being unnecessarily <code>!Send</code> due to storing a reference to a <code>!Sync</code> type across a yield point which can make things a bit awkward. However, the internal state of a generated future is entirely opaque, and can only be interacted with via <code>poll</code> which requires a mutable reference. So it seems like we could potentially have the compiler manually apply <code>Sync</code>/<code>Send</code> implementations in these cases. That would mean we couldn't ever add e.g. <code>Debug</code> impls that show the fields, but that doesn't seem like a thing we'd do anyway.</p>



<a name="221358786"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221358786" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221358786">(Jan 01 2021 at 16:44)</a>:</h4>
<p>Though I guess the reference-to-!Sync is probably not something the compiler would be able to figure out</p>



<a name="221358793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221358793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221358793">(Jan 01 2021 at 16:45)</a>:</h4>
<p>It could minimally have an unconditional <code>Sync</code> impl on the future itself though</p>



<a name="221375788"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221375788" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221375788">(Jan 02 2021 at 00:08)</a>:</h4>
<p>Making the futures <code>Sync</code> seems like an option</p>



<a name="221375792"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221375792" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221375792">(Jan 02 2021 at 00:08)</a>:</h4>
<p>I don't see how we could make them <code>Send</code></p>



<a name="221415899"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221415899" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mark Drobnak <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221415899">(Jan 02 2021 at 20:27)</a>:</h4>
<p>I wonder if the sync_wrapper crate is relevant here? <a href="https://docs.rs/sync_wrapper/0.1.0/sync_wrapper/struct.SyncWrapper.html">https://docs.rs/sync_wrapper/0.1.0/sync_wrapper/struct.SyncWrapper.html</a></p>
<p>I also just ran across <a href="https://docs.rs/send_wrapper/0.5.0/send_wrapper/">https://docs.rs/send_wrapper/0.5.0/send_wrapper/</a>, but haven't looked into it yet.<br>
Edit: it just does a runtime check to ensure the value's only accessed on the original thread.</p>



<a name="221416263"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221416263" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mark Drobnak <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221416263">(Jan 02 2021 at 20:36)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116883">tmandry</span> <a href="#narrow/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F/near/221375792">said</a>:</p>
<blockquote>
<p>I don't see how we could make them <code>Send</code></p>
</blockquote>
<p>I think the intuition behind it being <code>Send</code> is that the <code>!Sync</code> reference refers to data owned by a higher-up future in the future chain. i.e, it would be safe to move the higher-up future across threads, but moving just the lower future wouldn't be safe.</p>



<a name="221417131"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221417131" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221417131">(Jan 02 2021 at 21:01)</a>:</h4>
<p>I don't follow how <code>Sync</code> even applies to Futures (aparent from the unfortunate Hyper <code>Stream</code> which requires <code>Sync</code> for an unkown reason). When would one ever try to <code>poll </code> a <code>Future</code> from more than 1 thread?</p>



<a name="221417416"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221417416" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mark Drobnak <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221417416">(Jan 02 2021 at 21:07)</a>:</h4>
<p>Even if a future is <code>Sync</code>, it can't be polled from multiple threads because polling requires <code>&amp;mut</code> (so there can only be one reference to the future at the time of polling).</p>
<p>I'm not aware of a concrete reason for a future to want to be <code>Sync</code>, but perhaps it would be useful for load balancing a multi-threaded executor.</p>



<a name="221417471"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221417471" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mark Drobnak <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221417471">(Jan 02 2021 at 21:08)</a>:</h4>
<p>The main issue is if the <em>internal</em> data of a future is <code>Sync</code> or not. If the future holds a reference to a <code>!Sync</code> type then the future becomes <code>!Send</code>.</p>



<a name="221480745"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221480745" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221480745">(Jan 04 2021 at 01:04)</a>:</h4>
<p>yeah, to be clear, I don't think making them <code>Sync</code> would be very useful</p>



<a name="221480861"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221480861" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221480861">(Jan 04 2021 at 01:08)</a>:</h4>
<p>though some kind of work-stealing executor might want that, in fact</p>



<a name="221481001"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221481001" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221481001">(Jan 04 2021 at 01:12)</a>:</h4>
<p><span class="user-mention silent" data-user-id="218805">Mark Drobnak</span> <a href="#narrow/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F/near/221416263">said</a>:</p>
<blockquote>
<p>I think the intuition behind it being <code>Send</code> is that the <code>!Sync</code> reference refers to data owned by a higher-up future in the future chain. i.e, it would be safe to move the higher-up future across threads, but moving just the lower future wouldn't be safe.</p>
</blockquote>
<p>I see now, thanks</p>



<a name="221481022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221481022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221481022">(Jan 04 2021 at 01:13)</a>:</h4>
<p>Yeah I was really hoping for more Send impls but it seems quite hard to actually do that without some pretty detailed analysis of how references are being created</p>



<a name="221481073"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221481073" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221481073">(Jan 04 2021 at 01:14)</a>:</h4>
<p>are the references you see from futures higher up (futures polling you), or futures you poll, or both?</p>



<a name="221481092"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221481092" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221481092">(Jan 04 2021 at 01:15)</a>:</h4>
<p>it does seem like a difficult analysis, though I'm wondering if we can leverage <code>Pin</code> in some way, maybe</p>



<a name="221481158"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221481158" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Steven Fackler <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221481158">(Jan 04 2021 at 01:16)</a>:</h4>
<p>Not sure - this coming off of <a href="https://github.com/tokio-rs/tokio/issues/2545">https://github.com/tokio-rs/tokio/issues/2545</a></p>



<a name="221481525"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221481525" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221481525">(Jan 04 2021 at 01:26)</a>:</h4>
<blockquote>
<p>though some kind of work-stealing executor might want that, in fact</p>
</blockquote>
<p>No. It needs them to be <code>Send</code> - not <code>Sync</code>. Even if <code>Future</code>s get stolen, they will just be polled from one thread at a time</p>



<a name="221481686"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221481686" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221481686">(Jan 04 2021 at 01:30)</a>:</h4>
<p><span class="user-mention silent" data-user-id="243558">Steven Fackler</span> <a href="#narrow/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F/near/221481158">said</a>:</p>
<blockquote>
<p>Not sure - this coming off of <a href="https://github.com/tokio-rs/tokio/issues/2545">https://github.com/tokio-rs/tokio/issues/2545</a></p>
</blockquote>
<p>Isn't that more of a tokio issue than a language issue? tokio could offer a feature for spawning tasks which are guaranteed not be be migrated. It does with <code>LocalSet</code>, but that's not super convenient to use since it doesn't pin but create a fully independent runtime. I proposed having a way to pin resources on the existing runtime (e..g through task groups or configs), but so far this didn't see too much interest and support</p>



<a name="221495611"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221495611" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221495611">(Jan 04 2021 at 07:26)</a>:</h4>
<blockquote>
<p>No. It needs them to be <code>Send</code> - not <code>Sync</code>. Even if <code>Future</code>s get stolen, they will just be polled from one thread at a time</p>
</blockquote>
<p>In the scenario I'm imagining the tasks themselves won't move, since they're already pinned. If each thread is scoped to the executor lifetime it could "send" a mutable reference to a task across threads, which only requires <code>Sync</code>.</p>



<a name="221495685"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221495685" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tmandry <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221495685">(Jan 04 2021 at 07:28)</a>:</h4>
<p>even though no synchronization is really happening on the task itself</p>



<a name="221547578"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187312-wg-async-foundations/topic/Can%20async%20%7B%7D%20futures%20be%20Sync/Send%20more%20often%3F/near/221547578" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mark Drobnak <a href="https://rust-lang.github.io/zulip_archive/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F.html#221547578">(Jan 04 2021 at 16:43)</a>:</h4>
<p><span class="user-mention silent" data-user-id="204219">Matthias247</span> <a href="#narrow/stream/187312-wg-async-foundations/topic/Can.20async.20.7B.7D.20futures.20be.20Sync.2FSend.20more.20often.3F/near/221481686">said</a>:</p>
<blockquote>
<p>I proposed having a way to pin resources on the existing runtime (e..g through task groups or configs), but so far this didn't see too much interest and support</p>
</blockquote>
<p>I'm also interested in having  the core runtime workers support a local task queue, but I think <a href="https://github.com/tokio-rs/tokio/issues/2545#issuecomment-753450748">Darksonn's comment</a> about the increased scope of <code>block_in_place</code>'s footgun is the main reason the tokio maintainers don't want to use that approach.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>